mathematics Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...

, an information source is a sequence of

random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...

s ranging over a finite alphabet Γ, having a

stationary distribution Stationary distribution may refer to: * and , a special distribution for a Markov chain such that if the chain starts with its stationary distribution, the marginal distribution of all states at any time will always be the stationary distribution. ...

. The uncertainty, or

entropy rate In the mathematical theory of probability, the entropy rate or source information rate is a function assigning an entropy to a stochastic process. For a strongly stationary process, the conditional entropy for latest random variable eventually ...

, of an information source is defined as :

H\ = \lim_ H(X_n ,  X_0, X_1, \dots, X_)

where :

X_0, X_1, \dots, X_n

is the sequence of random variables defining the information source, and :

H(X_n ,  X_0, X_1, \dots, X_)

is the conditional

information entropy In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential states or possible outcomes. This measures the expected amount of information needed ...

of the sequence of random variables. Equivalently, one has :

H\ = \lim_ 
\frac.

References

* Robert B. Ash, ''Information Theory'', (1965) Dover Publications. zh-yue:資訊源 Information theory Stochastic processes {{statistics-stub

See also

References